testingiauh912fandomcom-20200214-history
TEST METHODS- Sareh Saatian
'Test methods ' What is inpute? Input consists of the new information contained in a given test task ,to which the test taker is expected to response . What is expected RESPONSE? Expected RESPONSE: 1) is a part of the test method. 2) WE can specify it through our test design. Input and expected response affect performance on language tests. Input contains: A) FORMAT of language B) NATURE of language Expected response contains: A) TYPES of restrictions on responseB) DEGREE of restriction on response The distribution of the new information must be processed and manipulated in order for the test taker to successfullycomplete a given test task. It divided into two parts: A) Compact B) DIFFUSE -new information which is distributed over a short time may be called: compact -new information which is distributed over a long time may be called: diffuse Example 1: listening to a lecture and then summarizing its content, requires keeping in mind information from beginning, the middle and the end, it is an example of diffuse Example 2: in a group oral interview consisting of fairly rapid exchanges, with each of the participants speaking oneor two sentences in turn, the new information in each utterance is likely to be compact. The type of information in input and expected response can be classified along three dimensions: 1) Concrete /abstract, 2) positive/negative 3) factual/counterfactual Abstract information: represent symbolic or linguistic. For example standard deviation, it using mathematical symbols Concrete information: is visual, auditory, tactile Negative information: Information can also vary in terms of the degree to which it is negative. Information presented with negative markers can vary in both the level and number of elements negated. Positive information: Information that includes no explicit negative elements or markers can be considered to be positive. Counterfactual information: is about conditions which are possible or probable in some alternative world, but not in the known, factual world (van Dijk 1977:79). Counterfactual sentences typically take the form of conditionals, such as: ‘If cows could jump over the moon, dogs might laugh. We might hypothesize that counterfactual information would be more difficult to process than factual information, and would be more difficult to interpret than those in which the factual and possible worlds are quite similar. Topic: Topic is generally understood as ‘what it is about’. The topic of the input in tests is determined by the test writer, who typicallychooses topics that she feels will be interesting and relevant to test takers. the topic of the response is expected to match, or berelevant to that of the input. In tests in which there is an interactive relationship between input and response, it is possible for the topic of the discourse to be determined jointly by the test administrator and the test taker. The topic of input has an important effecton test takers’ performance. if the topic is familiar to some test takers and not others, these individuals may have an unfair advantage , resulting in better performance.’ Clearly, we want to avoid this as a source of “test bias “. On the other hand, we alsowant the subject which is interesting and relevant to test takers test writers may select topics that are very general, and thusentirely unengaging to test takers. One the other hand, topics that are expected to be engaging because they are interesting and relevant in favor of some test takers. One possible solution to this problem is the presentation of a fairly large number of topics, which could reduce serious bias. Genre: According to: 1) HYMES (1972a) the term ‘genre’ refers to ‘formal characteristics that are traditionally recognized’, giving as examples the categories of ‘poem, myth, tale, and proverb. 2) COULTHARD(1977) a genre is one type of stylistic structure for organizing sentences and utterances into larger units such as greeting, farewell and prayers. 3) BROWN and Yule (1983) suggest that genres may differ in their formal characteristics such as paragraph structure and the distribution of sentences types. if the language of the input in a given test is characteristic of a genre that is unfamiliar to the test taker, we would hypothesize that tasks that depend on the interpretation OF THAT input would be relatively difficult. Unfamiliarity withthe characteristics of a given genre may also make the expected response more difficult. Organizational characteristics: Organizational competence comprising those abilities that are related to controlling the formal organization of language. This formal organization is a characteristic of both the input and the response. It is in three types: 1) Grammar 2) Cohesion 3) Rhetorical The input material is organized in terms of grammar, cohesion and rhetorical Structure, and in tests that require a selected response it may include errors involving one or moreorganizational characteristics. An example of input involving grammatical errors is a multiple-choice item in which several words in a sentence are underlined, one of which is ungrammatical. Pragmatic CHARACTERISTICS: The input and response can be characterized in the term of the language functionperformed. Input and the response can involve the performance of the variety of a language function.an example of this is a oral interview in which the examinee isinvolved in the conversation that hi virtually forgets the formal organizational characteristics of the discourse. Determining the functionality of a given test taskis problematical, because it depends on the interaction between the test taker andthe task material . Sociolinguistic characteristic: It consistsOF: Dialect or variety- Register- Naturalness- Dialect or variety While we need to test an individual’s ability to use different dialects or varieties of a given language appropriately, in the majority of language testing situations the target language to be measured is in some sense regarded as a ‘standard’ dialect or variety. We need to make every effort to ensure that the dialect or variety of language of both the input and the expected response is appropriate to the language use context in which test takers are most likely to find themselves using the language, or which is recognized by language educators as the most appropriate variety. For example, to the extent that the dialect of language used and taught in schools is different from students’ native regional. In India, for example it may be appropriate to use Indian English as the test language in some contexts and British English in others. In testing second or foreign language proficiency, the variety of language used in the test needs to be appropriate to the needs of the learners. Thus, for American college students studying Spanish, it may be that a variety of Spanish spoken in Latin America is appropriate, while Castilian Spanish might be more appropriate for students of Spanish in England. Register: (formal/informal) the register of test language needs to be appropriate to the context and needs of rest takers. In many tests, particularly those designed to be used in educational settings, the register of input and expected response is formal. When the register of the test language is not entirely appropriate to the register of the target language use CONTEXT, the test results may provide misleading information. Language use Bloom and Lahey (1987) divided language into three separate but overlapping components : content , form , use According to : 1) Function of LANGUAGE: language USE refers to the reasons why people communicate. 2) Context of LANGUAGE: language USE refers to how people both understand and choose from alternative linguistic forms in order to reach their goals. Restrictions on expected response Virtually all language use is restricted in various way by the context or situation. For example, Upon meeting a colleague on the way to class, we are usually not free to utter whatever may come into our head, but are restricted our perform (greeting) and the register we use(formal, informal). There are five different areas in which restrictions can be placed on expected responses in language testing situations: 1) Restrictions on the channel. (2) Restrictions on the format. (3) Restrictions on organizational characteristics (4) Restrictions on propositional and illocutionary characteristics. (5) Restrictions on the time or length of the response. Restrictions on channel Language use takes place under a wide variety of conditions. Conversations are carried out face-to-face, in quiet rooms, and on noisy streets. Phone conversations take place out of visual contact over lines. In fact, many types of language use normally TAKE PLACE under less than ideal conditions. for example, conversation with taxi driver. In many language testing situations, suchinterference in the channel does not occur and, therefore, cannot influence the performance of the participants. Restrictions on FORMAT: Insome tests the format is highly restricted, as in selection or identification response types, while in other tests the format may be fairly unrestricted, as in A COMPOSITION test requiring the production of a writing sample. Restrictions on organizational characteristics It divided into two parts: 1) Restrictions on grammatical forms: Restrictions on form at the word level are common in language testing. The most obvious example is the multiple-choice vocabulary item in which test takers are required to deal with the meanings of a specific pair of words, without the opportunity to express the equivalent meaning in their own words. Restrictions on form at the sentence level occur in tests, where the materials are designed to elicit particular structures. 2) RESTRICTIONS on the organization of DISCOURSE: A high level of restriction on language performance is in the organization of discourse. An example of such a restriction would be requiring that a student follow a specific rhetorical pattern , such as comparing and contrasting two things. Another example is the oral interview , in which the subject is told to pretend to call up a close friend and converse with him on the telephone. The organization is restricted by instructing the subject to do four things: first, inquire about the friend’s health and the health of the members of his family; second, find out what the friend had been doing; third, suggest that the two of them do something together that evening; and finally, make plans for how to get where they have decided on going (Bachman and Palmer 1983a). Restrictions on propositional characteristics Restrictions may also apply to the propositional content. Consider the meaning of the following utterance: ‘How many Children do you want to have?’ The propositional content is about family size. Consider the illocutionary force of this question, were it to be posed unexpectedly by a young man on his first date with a young woman. His intention might well be to increase the intimacy of the relationship. The effect, of course, might be to break up the relationship, if the woman were not at allinterested in having children. Restrictions on time or length of RESPONSE: Language use is always constrained, by time or length. In testing situations, administrative considerations almost plac additional restrictions on time or length. Relationship between input and response It can be classified into three types: Reciprocal input and response- Nonreciprocal input and response- Adaptive input and response- Reciprocal input and response: Reciprocal language use can be defined as the use of language by one individual to produce an effect in another individual through the reductionof uncertainty with knowledge of results. This definition contains a number of components. The term ‘language’ focuses on the verbal component (spoken or written) ,The phrase ‘one individual’ and ‘another individual’ make explicit the requirement that at least two parties - a sender and a receiver - be involved. The phrase ‘to produce an effect’ indicates that the language use has a communicative goal. The phrase ‘reduction of uncertainty’ characterizes the means by which the communicative goal is AFFECTED. One of the distinguishing characteristics of reciprocal language use, is INTERACTION, the second distinguishing characteristic of reciprocal language use, is the presence of feedback Nonreciprocal input and response: Nonreciprocal language use is that in which there is no interaction between language users, that is, in which continual give and take between two or more individuals does not take place. In nonreciprocal language use, therefore, there is no feedback. In reading a book, for example, the reader is seldom able to give feedback to the author, and the text of the book will not change. Other examples of nonreciprocal language use include letter writing , reading, listening to lectures, watching movies and talking to oneself. Nonreciprocal test tasks, are those in which both interaction and feedback are missing, as in a cloze or dictation test. language use in general is often nonreciprocal, as in reading books and newspapers or listening to radio and television programs. Adaptive input and response If the input is influenced by the response, but without the feedback, then the relationship between input and response is adaptive. In an adaptive, the particular tasks to the test taker are determined by her response to previous tasks. For example an individual taking an adaptively administered multiple-choice test, typically begins with an item that is of medium difficulty. If she answers this item correctly, the next item presented will be slightly more difficult. In most adaptive tests, the test taker is not aware of the correctness of her responses. The difference between adaptive and reciprocal test is: Adaptive tests do not involve the feedback or reduction of uncertaintythrough knowledge of results that characterizes reciprocal language tests. An oral interview is reciprocal. Multiple-choice items is not reciprocal, since the test taker may be totally unaware of the effect of her responses on the selection of test items, and receives no feedback on the correctness of he answers.